Graphics rendering refers generally to the process of generating a two-dimensional image from graphical models. A graphical model defines attributes of a real or imaginary object which are to be represented in a rendered image. These attributes include, for example, color, shape, and position of an object in a graphics scene. In the process of rendering these models, a graphics system generates a display image, which typically includes an array of pixel data. A pixel is a point or picture element in a display device, and in the context of graphics processing, also corresponds to a point in the two-dimensional space to which the graphical models are rendered. The pixel data of a rendered image includes one or more data values describing attributes of a pixel used to display it. For instance, this pixel data can include intensity values for color components. These intensity values are then used to display the array of pixels comprising an image on physical output device such as a raster display device.
Graphics processing is often classified by the dimension of the models to be rendered 25 to an image. For instance, two-dimensional graphics processing ("2-D graphics") refers to the generation of an image from graphical models having two dimensions (x and y coordinates). and three-dimensional graphics processing ("3-D" graphics) refers the processing of three-dimensional models.
Graphics processing can also be classified as "real time" which means that 1) the display image is updated so that the user perceives continuous motion of the objects in the scene; and 2) there is minimal and predictable "transport delay" between user input, which change the position of objects or the viewpoint of the scene, and the display of an image in response to this input. To achieve this affect, a scene, or objects in the scene, must be rendered within a predefined period of time.
In 3-D graphics applications, an objects in a scene are represented by 3-D graphical models, which include geometric data used to model the surface and position of the objects, and visual attributes used to model the appearance of the objects. There are a number of ways that a geometric model can represent a 3-D object, including polygon meshes, parametric surfaces, or quadratic surfaces. Using a polygon mesh, for example, the surface of an object is modeled with several interconnected polygons. The surface elements, in this case polygons, are referred to as geometric primitives. Visual attributes such as red, green, and blue color data, and possibly other model data is stored at the vertices of the polygon.
In the rendering process, the geometric primitives corresponding to objects in a scene are processed to generate a display image. In the context of 3-D graphics, the rendering process includes transforming the graphical models in a scene, and rasterizing the geometric primitives in the models to generate pixel data.
The rendering process typically begins by transforming the vertices of the geometric primitives to prepare the model data for the rasterizing step. While the specific details of the transformation phase varies, a few examples will illustrate the process. The modeling transform, in some systems, is used to convert the vertices of a model from the model's local coordinate to world coordinates, the coordinates in which a complete scene is represented. The next step is to determine potentially visible objects in a 3-D space referred to as the view volume. This step is commonly performed in view reference coordinates, which describes object locations relative to a viewpoint or eyepoint. Objects that are not potentially visible at this stage can be disregarded while objects that are least partially in the view volume are "clipped" to the view volume.
After transforming the objects, the geometric primitives for the objects are "rasterized." Rasterizing generally refers to the process of computing a pixel value for a pixel in the view space based on data from the geometric primitives that project onto or "cover" the pixel. Rasterizing is sometimes referred to as "tiling" because of the analogy to tiling a floor. Imagine that the pixels are square elements or tiles, and that a polygon is the floor plan. The rasterizing step includes tiling this floor plan by computing pixel values for the pixels or "tiles" within the polygon.
While there are a number of ways to scan convert a geometric primitive, this process generally involves computing a pixel intensity value or values based on the data from polygons covering that pixel. For example, color values stored at the vertices of a polygon can be interpolated to find a color value at a given pixel in the interior of the polygon. During this process, lighting and shading models can also be used to compute pixel values for pixels across the surface of the polygon.
The color and opacity of a pixel can also be determined using a process called texture mapping. Texture mapping is the process of mapping a two-dimensional image to the surface of an object. Where the surface of the object is modeled with polygons, texture mapping involves mapping the image to the surface of a polygon.
In one application of texture mapping, colors accessed from a two-dimensional table (2-D table) are applied to the surface of a polygon. The table lookup is computed by interpolating 2-D texture indices specified at each vertex, and using the interpolated index to look up the color at each pixel.
In some cases, a polygon may have translucency. The translucency of a polygon is typically represented by an opacity value, specifying the degree to which the polygon is translucent. Translucency, while more complicated, can be handled in a similar fashion to color. For example, each pixel rasterized within a polygon can have a color and opacity formed by some function of a color and opacity value interpolated over the polygon from values at its vertices, and a color and opacity retrieved from the texture map. The function that combines the interpolated and looked-up color/opacity values is sometimes referred to as the texture application function. The result of the texture application function is called the composed color and composed opacity.
An example of a texture application function called the "modulate mode" is described by J. Neider, T. Davis, and M. Woo in "OpenGL Programming Guide", Addison-Wesley Publishing Co. Reading, Mass., 1992, pp. 275.
In the OpenGL modulate mode, the composed color and opacity, C.sub.c and A.sub.c, are given by:
C.sub.c =C.sub.i C.sub.t PA1 A.sub.c =A.sub.i A.sub.t PA1 D N.multidot.L+E PA1 1. [buffer color=D N.multidot.L] Render polygon interpolating N.multidot.L, and using D as the texture map. The texture application function multiplies the interpolated and looked-up quantities. The blending function simply stores the composed color as the buffer color. PA1 2. [buffer color=D N.multidot.L+E] Render polygon interpolating the unit color and using E as the texture map. This effectively ignores the interpolated color and yields E as the composed result. The blending function is set to add this result to the result of the previous pass.
where C.sub.i and A.sub.i are the color and opacity interpolated over the polygon, and C.sub.t and A.sub.t are the color and opacity retrieved from the texture map.
A major problem in the design of a graphics system is designing the system to support complex lighting add shading operations. There is a conflict between simplifying the hardware design on the one hand, and supporting more sophisticated lighting operations on the other. One way to simplify the hardware design is to support only a single texture map access for each pixel rasterized. Unfortunately, more complex lighting models often require that more than one texture map be accessed.
One way to address this problem is to decompose the lighting model into a series of passes, each of which uses a single texture map. The results of subsequent passes can then be combined with the results of a previous pass using a function called the blending function. The result of the blending function is called the buffer color and buffer opacity because it is stored in the rasterization buffer.
Blending functions from OpenGL involve linear combinations of the interpolated and texture-mapped color/opacity. For example, consider a polygon that has a texture-mapped diffuse color as well as a separately texture-mapped emission color:
where D is the texture-mapped diffuse color, N.multidot.L is the dot product of the normal with the light direction, and E is the texture-mapped emission color. The following two passes decompose this lighting model into steps the hardware can compute:
When a general lighting expression is decomposed into a series of passes, the same geometric primitives are rasterized multiple times. Each time the geometry is rasterized, the different terms of the lighting expression are accumulated.
As noted above, it is particularly difficult to implement multi-pass rendering in a graphics system. One way to support a limited form of multi-pass rendering is to use a single accumulation buffer. The single accumulation buffer is discussed in P. Haeberli and K. Akeley, "The Accumulation Buffer: Hardware Support for High-Quality Rendering," in Computer Graphics, SIGGRAPH '90 Proceedings, August, 1990, 24(4), pp. 309-318. While the Accumulation Buffer can be used for multi-pass rendering, it only addresses a limited subset of possible lighting expressions.
One major limitation to using a single accumulator is that it cannot support multi-pass rendering operations where the result of one pass must be stored temporarily and then added or multiplied with results of another pass. The problem is analogous to compiling arithmetic expressions using an ALU with a single accumulator. To compute an expression like a*b+c*d requires at least one temporary variable (e.g., to store a*b). In the same way, a single accumulation buffer is not sufficient for all lighting expressions because there is only a single destination for intermediate expressions. The accumulation buffer provides no support for temporarily storing an intermediate result, which can then be combined with results of other passes.
In designing a graphics system to support multi-pass rendering, it is important that the problem of aliasing also be addressed. From the tiling analogy above, it is clear that discrete pixels cannot precisely represent continues surfaces. For example, a polygon may only partially cover a pixel region. In this case, the edge or edges of a polygon cross over the pixel region. If the pixel were approximated as being fully covered by this polygon, anomalies such as jaggy edges in the rendered image would likely result. A technique known generally as antialiasing attempts to address this problem. In general, antialiasing is used to compute pixel intensities for partially covered pixels to reduce the discontinuities introduced by representing a continues object with a discrete array of pixels.
One way to perform antialiasing is to generate pixel fragments for partially covered pixels during the rasterization process and then resolve the pixel fragments stored for each pixel to compute final color values for each pixel. A software implementation of this approach is discussed in: L. Carpenter in "The A-Buffer, An Antialiased Hidden Surface Method," in Computer Graphics, SIGGRAPH '84 proceedings, July 1984, 18(3), pp. 103-108.
It is very difficult to support this approach to antialiasing and multi-pass rendering at he same time because of the problem of blending pixel fragments. This problem can arise when attempting to make more than one rendering pass before resolving the pixel fragments. For example, when rendering the same geometry in two passes, the pixel fragments from the second pass need to be combined with the pixel fragments stored from the first pass. This problem has not been addressed.
As outlined above, there are a number of difficulties in supporting advanced lighting and shading operations in a graphics rendering system. It is even more difficult to support these features in a real time, interactive graphics system. Sophisticated rendering features such as multiple light source shadows, environment mapped reflective objects, spot lights, and ground fog have been limited to off-line software rendering systems. Because of the complexity of the computations involved, such features are usually attempted in software only and do not achieve real time performance.
Present real-time graphics rendering systems do not adequately support advanced lighting and shading functions. The rigid rendering pipelines and fixed rendering modes of high end 3-D graphics workstations make it impossible to implement these advanced effects without drastic reductions in real-time performance. As a result, users who require real-time display must put up with limited rendering flexibility.